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Chapter I 



Background 



A. Policy Problem and Research Questions 

Public school teachers are usually paid according to two objective criteria: their years of 
experience and their educational attainment (certificates, degrees, or coursework). This 
system, known as the uniform salary schedule, has received criticism for its unfairness, its 
failure to reward effort or skill, and its inefficiency in not encouraging hard work or talent 
(Hanushek 1981). 

Education policymakers seeking to reform the system of teacher compensation have 
tried many times, often without success, to tie teacher compensation more closely to the 
quantity and quality of teachers’ work. An influential 1983 report by the National 
Commission on Excellence in Education, entitled A Nation at Risk, spotlighted the problem 
and spurred a wave of reform during the mid- to late 1980s. Many of the reforms included 
career ladders for teachers. Career ladders allow teachers to advance in salary based on 
factors other than seniority, such as demonstrated skills or performance. Most of the 
reforms enacted in the mid- to late 1980s did not last long (Glazerman 2004), but the present 
study focuses on one exception: a teacher career ladder program started in Missouri in 1986 
that continues to operate today more or less unchanged. 

Missouri’s Career Ladder program set forth two primary goals: to improve student 
achievement and to attract and retain effective teachers. The program offers opportunities 
for teachers to earn extra pay for extra work and professional development, with eligibility 
for these opportunities based on a combination of seniority and subjective performance 
evaluation. It is the intent of Missouri policymakers to improve academic services, 
programs, and student learning outcomes both by encouraging teachers to engage in 
productive activities and by attracting and retaining effective teachers through the extra pay 
associated with those activities. 

1. Research questions 

This report is one of three from a study conducted by Mathematica Policy Research, 
Inc. (MPR) of the Missouri program, which posed the following broad research questions: 
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1 . How does the program operate in theory and in practice? 

2. What effect does the Career Ladder have on student achievement? 

3. What effect does the Career Ladder have on teachers’ career decisions, 
specifically their decision to remain in their district or in teaching? 

This report directly addresses the first research question while providing some teacher 
and district perspectives on the second and third research questions. Two companion 
reports (Booker and Glazerman 2008a; 2008b) address the second and third questions by 
providing a quantitative analysis of the relationships between Missouri’s Career Ladder and 
student and teacher outcomes. The goal of the present report is to document everything we 
were able to learn about the program that might help readers interpret the two companion 
reports. We use qualitative data to understand how districts came to be in the program, how 
teachers decided to participate, and how teachers qualified for and earned payments. We 
also explored the mechanics of the program, its oversight and accountability, and the 
possible pathways by which it could produce beneficial outcomes. 

2. Data 

This report draws on the following three types of data: 

1. We reviewed all the relevant official program documentation that might tell 
us how the program works, including state legislation, regulations issued by the 
Department of Elementary and Secondary Education (DESE), and other 
program documents, such as the set of annual reports issued by DESE. We 
also queried a longitudinal dataset that DESE provided covering teacher and 
district participation in the program since 1991. 

2. We conducted four focus groups with teachers who were participating in 
Career Ladder. We conducted each focus group in a separate school district, 
selected to provide diversity of settings (urban, suburban, rural). It is important 
to keep in mind that the focus groups were designed to elicit stories about how 
teachers came to participate in the program and how they might have 
experienced it, not to generalize about all teachers who ever took part in Career 
Ladder. Therefore we report on experiences without particular regard to the 
number or percentage of teachers who had each particular experience or 
impression. We spoke with teachers in grades K-12 who were at various stages 
of the Career Ladder. 

3. We conducted telephone interviews with officials from 15 randomly selected 
participating school districts. The interviews were aimed at understanding how 
districts came to participate in the program and at how program operations 
look from a district level perspective. We intended to interview a district 
administrator, but always asked to speak with the most knowledgeable person 
on the district’s Career Ladder participation. In most cases, this led us to the 
person who chaired a local Career Ladder committee, often a senior teacher. 
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Appendix A describes the data and methods used for this study. The telephone 
interview guide is attached as Appendix B and the focus group protocol as Appendix C. 

3. Previous research 

To date, policymakers have had little evidence on which to base answers to the above 
questions about Missouri’s Career Ladder program. The only evidence that we were able to 
find on the effectiveness or even the operation of the program was limited to two reports on 
early program implementation (Schofer et al. 1987; Taylor and Madsen 1989), two single- 
district studies from the program’s early years (Ebmeier and Hart 1992; Henson and Hall 
1993), and a brief set of tabulations by the Missouri Department of Elementary and 
Secondary Education (DESE) that used 1999 test score results for a subset of the state’s 
districts (Phillips 2000). Our qualitative analysis aims to understand program 
implementation. We drew on teacher perceptions of career effects elicited in focus groups 
with participating teachers as well as on interviews with program administrators, thereby 
producing a comprehensive picture of how Missouri’s Career Ladder program operates and 
how it has evolved over more than two decades. 

B. Overview of the Missouri Career Ladder Program 

For background purposes, we describe the program as it operates in theory according to 
available program documents and published literature. Later chapters of the report examine 
program operations in more detail, presenting relevant information based on first-hand data 
collected for this study. 

Program Structure and Operations. Through the Career Ladder program, teachers 
who meet statewide and district performance criteria are eligible to receive supplementary 
pay for Career Ladder responsibilities, which may take the form of extra work or 
participation in professional development activities. The program supplements the regular 
salary schedule but does not replace it. Career Ladder responsibilities must be academic in 
nature and directly related to the improvement of student programs and services. Career 
Ladder involves three stages based on years of experience and other factors. To move up 
the ladder, teachers must undergo an assessment at each stage through periodic observations 
and evaluations of documentation. Each successive stage offers the opportunity to receive 
more supplementary pay for Career Ladder responsibilities: up to $1,500 for Stage I, $3,000 
for Stage II, and $5,000 for Stage III. Out of more than 65,000 teachers in 524 districts 
statewide, more than 17,000 teachers (26 percent) from 333 districts (64 percent) 
participated in Career Ladder during the 2005-2006 school year. 

The Missouri program is distinctive among the nation’s teacher compensation reforms 
in that it is the most mature program. It has been operating since 1986, outlasting dozens of 
programs that were introduced around the country at the same time. In addition, the 
Missouri Career Ladder is unusual in how it mixes teacher performance, tenure, and extra 
responsibilities to define salary supplements. Teachers must advance along the Career 
Ladder based on tenure and progress in performance as rated by classroom observers, yet 
they receive bonuses for taking on extra responsibilities. 
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District Participation. Missouri’s program operates statewide, and districts must 
choose to participate and provide matching funds. Districts interested in implementing a 
Career Ladder program must submit a District Career Ladder Plan (DCLP) to DESE, which 
is responsible for approving plans that meet state guidelines for improving student academic 
services and programs. Districts share the cost of the program with the state. Poorer 
districts as well as smaller districts receive a higher percentage of state matching funds. The 
three possible matching rates are 60/40, 50/50, and 40/60. 

Teacher Eligibility and Qualifications for a Bonus. To enroll in Career Ladder and 
qualify for bonuses, teachers must develop a Career Development Plan (CDP) associating 
each Career Ladder responsibility with either a designated plan or some other instructional 
improvement. The teacher then submits the CDP to the district Career Ladder Review 
Committee, which is made up of educators (selected by teachers) and administrators and 
charged with approving CDPs. 

To receive a salary supplement, teachers must spend a specified amount of time on a 
certain number of responsibilities outside of their contracted time. Examples of extra 
responsibilities assumed by Career Ladder teachers include extra work — providing students 
with opportunities for enhanced learning experiences, remedial assistance, and various 
extended day/year activities — and professional development activities — participation in 
professional growth activities, including college classes, workshops, and professional 
organizations. 1 Teachers must also meet milestones for years of teaching and program 
experience, along with certain performance benchmarks on the state Performance-Based 
Teacher Evaluation (PBTE) system. The district’s Career Ladder Review Committee 
evaluates the teachers to determine if they have carried out their responsibilities and should 
receive supplementary pay. Except for the case of the Kansas City and St. Louis school 
districts, which operate their own retirement benefits plans, supplemental pay counts as 
salary for the computation of retirement benefits throughout the state. 

Drawing on a series of interviews with district leaders and other stakeholders as well as 
on the results of focus groups with teachers, the remainder of this paper examines how 
Career Ladder has operated in practice. We discuss program operations at the district level 
in Chapter II. Chapter III reports on the teacher-level analysis of program operations. 
Chapter IV discusses the mechanism by which Career Ladder might have impacts. Chapter 
V offers some conclusions and discussion of conclusions and their implications for policy. 



1 DESE recommends that teachers should not spend more than one-third of Career Ladder hours on 
college classes and workshops. 
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Chapter II 



The Role of the District 



T he Missouri Career Ladder is a state-level policy with funding and guidelines 
emanating from the state government, but it is locally implemented, with school 
districts deciding to participate and making decisions that shape how the program 
truly operates. This chapter explores the role of the district in relation to the state. Given 
that the decision to participate in the Career Ladder program necessarily first rests with the 
school district, we sought to understand how some Missouri districts came to participate in 
the program. The participation decision explains whom the program does and does not 
serve, helps illuminate district policymakers’ program expectations, and could help state 
policymakers understand how state support and rulemaking affect intended participants. 
We sought to explain which districts participate and why, but we also explored the oversight 
role of the state and the district’s own oversight body, called the Career Ladder Review 
Committee. 

A. The Participation Process 
1. District Career Ladder Plan 

To become eligible for state Career Ladder funding, a school district must first submit a 
District Career Ladder Plan to the state. The DCLP outlines the program’s purpose, 
compensation levels, eligibility requirements, guidelines for teacher performance, and 
guidelines for acceptable (reimbursable) program activities. 

The state guidelines for the DCLP specify a series of administrative guidelines and 
required tasks. Teachers, administrators, and community members are required to be 
involved in the development of the DCLP. Districts are directed to form a Career Ladder 
Review Committee, consisting of teachers and administrators, that is responsible for 
reviewing materials and approving payment to participating teachers. Under the direction of 
the school board and with assistance from assorted stakeholders, districts must also 
periodically review the Career Ladder program. 

The state provides districts with a model Career Ladder plan and most of the districts 
where we conducted interviews followed the model plan closely when crafting their own 
DCLP. Consequently, most of the district plans we reviewed for the study were similar to 
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the state template (and, by extension, to each other. However, some districts enacted 
additional eligibility requirements for teachers or restricted the activities eligible for 
compensation, making their requirements more stringent than the state guidelines. 

After submitting their initial DCLP, districts must resubmit their plan only if they 
modify it; otherwise, their existing plan remains in effect. Some district representatives that 
we interviewed indicated that the Career Ladder Review Committee periodically discusses 
whether to alter the allowable activities under the program, but they mentioned no other 
potential revisions to the DCLP. In some cases, districts have sharpened the focus on 
student achievement, subsequently requiring more work in activities such as tutoring. 

2. State Support and Oversight 

In addition to providing districts with guidelines and requirements for participation, the 
state has imposed other mandates on participating districts. By April 15 of every year, each 
participating district must provide the state with the exact number of participants for the 
following program year. Each district must then verify the count of participants midway 
through the school year. DESE stipulates other deadlines for confirming participants and a 
date for submitting a summary of Career Ladder program activities for the previous year. 

The state undertakes several activities to assist districts with Career Ladder operations. 
DESE conducts technical assistance visits during the fall to a subset of participating districts; 
in 2007, it scheduled 62 such visits,, with 76 scheduled in 2008. However, most interviewees 
mentioned that the visits involved a review of their Career Ladder plans rather than technical 
assistance. 

Our interviews with district and DESE representatives suggested that interaction 
between the district and the state was greater in the initial years of Career Ladder 
participation and then declined, perhaps reflecting a need to overcome start-up barriers. 
Some respondents indicated that the state previously offered districts assistance in the form 
of workshops during the summer. Most district staff thought that state oversight was 
effective; a number noted that DESE staff were readily available to answer questions and 
provide program guidance and that communication with the state was unproblematic. 

B. Participation Trends 

The Career Ladder program has steadily grown over the years. In recent years, the 
number of districts participating in the program has risen to 328, or 60 percent of districts in 
the state. The number has generally been rising, from 63 districts (about 18 percent of the 
state) in the program’s first year to 32 percent in 1991, the first year for which we have 
detailed data, to a peak of 62 percent in 2003, roughly the level at which participation 
remains today (see Figure II. 1). The sharpest increase in participation came during the early 
1990s. 
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Figure 11.1. District Participation Rates 



Percent of 
Districts Enrolled 




Year 



♦ Districts with Fewer Than 1 ,500 students 
M Districts with 1 ,500 to 5,000 students 
Districts with More than 5,000 students 
All Districts 



Once districts decide to participate, they typically do not discontinue participation. The 
only declines in the participation rate — in 1997, 2004, 2005, and 2007 — have totaled less 
than 2 percentage points, with some of the declines, including the most recent one (from 61 
to 60 percent), resulting from an increase in the number of districts in the state rather than 
from a decline in the number of participating districts. 

State union and government officials told us that Career Ladder was designed with the 
aim of helping raise salaries in mainly small, rural school districts, as is largely borne out by 
participation trends. Figure II. 1, which shows overall district participation trends since 1991 
also shows trends for different types of districts broken into categories of small, medium, 
and large based on student enrollment. While participation has reached nearly 70 percent 
for small districts (defined as those with fewer then 1,500 students), it has hovered closer to 
50 percent for medium-sized districts (having 1,500 to 5,000 students) and has remained 
under 30 percent for districts with more than 5,000 students. 



II: The Role of the District 



C. Explaining District Participation Decisions 



1. Who Decides? 

It may help policymakers understand why districts participate in Career Ladder to know 
that teachers are the driving force behind district participation decisions, according to people 
we interviewed. While school boards must allocate the local share of funds for participation, 
we often heard that teacher interest was the driving factor behind program participation. In 
one large district, the teacher’s union wanted the program. In two other districts, a group of 
teachers interested in the program persuaded the school board to vote for participation 
despite the board’s initial reluctance. In another district, the central administration — 
specifically, the superintendent — drove the participation decision, but teacher interest helped 
spur the decision. 

2. Rationale for Participating: What Did Districts Hope to Achieve With the 

Program? 

District representatives, many of whom were classroom teachers who had taken on 
special duties helping to mn the local Career Ladder program, cited numerous program 
benefits as the rationale for program participation. Interviewees from nearly all of the 
districts said that increased teacher compensation was the primary goal for instituting the 
program. Many added that the increased pay was a means to achieve other goals, such as 
increased teacher satisfaction, teacher retention, and, ultimately, student achievement. Most 
thought that increased tutoring or other activities would lead to achievement gains; some 
respondents also perceived such activities as an inherently beneficial, that is, regardless of 
any potential impact on achievement. Representatives from one district mentioned 
improved classroom instruction through professional and curriculum development as an 
initial factor motivating the participation decision. 

District representatives also cited cost factors that played into the participation decision. 
For districts, the cost is a primarily a function of the number of participating teachers and 
the level at which they participate — both of which determine the amount of bonuses to pay 
out — as well as the local matching rate. Districts may have other costs related to program 
administration, such as the initial effort of submitting and, if necessary, revising a DCLP and 
the ongoing efforts of evaluating teachers, reviewing plans, making local mle changes, and 
overseeing program operations. Next, we discuss the critical ingredient in determining the 
cost to a district, its matching rate for state funding. 

3. Matching Rate 

Perhaps the most important factor in a district’s decision to participate in Career Ladder 
is the generosity of the program, which is a function of the state matching rate. The higher 
the share of state funding per dollar of local funding, the more likely we expect that a district 
will participate, all other things equal. Matching rates vary by district, and the mles for 
determining matching rates have changed. 
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The matching rate formula in effect from program inception through the 1995—1996 
school year was based entirely on the district’s assessed property value per pupil. Districts 
were ranked on this measure from lowest to highest and divided into 12 groups. Districts 
with the lowest assessed property value per pupil received 90 percent state funding for 
program expenses. For each successive group of districts, the state funding share declined 
by 5 percentage points, with the district in the highest group receiving 35 percent state 
funding. 



Legislation enacted in 1996 significantly changed the funding formula by making the 
funding distribution narrower with fewer distinct matching rates; it also added total 
enrollment as a ranking factor. The current formula involves an initial sorting of districts by 
assessed property value per pupil, with the top 25 percent of districts on this measure 
categorized as Group 1. The remaining 75 percent of districts are then rank-ordered 
according to total enrollment from highest to lowest; the highest one-third of districts 
constitute Group 2, and the remaining two-thirds constitute Group 3. The first group 
receives 40 percent state funding, the second 50 percent, and the third 60 percent. Figure 
II.2 shows the current matching rate by district wealth (property value per pupil) and size 
(K-12 student enrollment), with the outliers “top-coded” (assigned to a maximum value of 
$80,000 per pupil and 10,000 students) to make the figure easier to read. We used 1986 
property valuation data and 1991 enrollment data for illustration because data from those 
years were most readily available. 



Figure 11.2. Matching Rate Factors 




High wealth: 
40% state match 



• • 



0 1,000 

Low wealth, 

low enrollment: 

60% state match 



2,000 3,000 



4,000 5,000 6,000 

Enrollment 



» *• • 4_ow wealth, high enrollment: 

• * 50% state match 



7,000 8,000 9,000 10,000 



Notes: Calculations from data provided by the Missouri Department of Elementary and Secondary Education. Each 

dot represents one school district. Property wealth values are top-coded at $80,000 per pupil; enrollment is 
top-coded at 10,000 students. 
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The revised funding formula reduced the influence of district property values per pupil 
while providing less state funding for districts with greater assessed wealth per pupil. It 
narrowed the range of state matching rates but added a component that gives weight to 
smaller, primarily mral, districts. To make the rate change more gradual, districts 
participating at the time of the legislative change and with a state matching rate higher than 
that specified by the new formula saw their state matching rate decrease by 5 percentage 
points per year until it reached the mandated level. 

The mle change might have driven districts away from the program, but no interview 
respondents reported any effects of the funding formula change on their district’s program 
participation. Several noted that the change had no effect whatsoever; others simply 
indicated that the district came up with any additional funds necessary to continue 
participation. For one district, the change caused the administration to re-evaluate the 
program and its costs/benefits; ultimately, the district decided to continue participation. 

The 1996 legislative change also contained a “grandfather” clause specifying that 
teachers participating in the program at the time of the change and continuing to participate 
in the program thereafter would have their Career Ladder payments matched at the same 
rate that was in effect before the change, thereby reducing district expenditures for certain 
teachers. When asked, no district reported any effects of this provision. 

D. Program Oversight within the District 

Oversight within the district is primarily the responsibility of each district’s Career 
Ladder Review Committee, whose members review teacher materials and approve payments 
to participants. According to DESE, the committee’s mandate is broader still. “The local 
committee is responsible for the quality of the program at the district level,” and its 
members “are required to adhere to the minimum standards established by” DESE. 2 The 
committee is also responsible for any additional standards that may be established. 

1. Composition and Leadership of the Career Ladder Review Committee 

The Career Ladder Review Committee is typically a teacher-led body composed of 
teachers and administrators, with some districts extending membership to community 
members or school board members. Committee members for districts in our sample 
numbered between 3 and 30, with an average of 1 1 members per district. Each committee 
was chaired by a teacher, except for one district’s committee, which was headed by an 
administrator (the superintendent). Typically, however, the administrator was a non- voting 
member. Teacher committee members were generally more experienced teachers than the 
overall pool of participants. 



2 DESE Career Ladder Frequently Asked Questions. Available online at: 
http:/ / dese.mo.gov/ divteachqual/ careerladder/FAQs.htm 
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2. Committee Responsibilities 

The committee’s primary responsibility is to oversee teacher-participants by reviewing 
teacher CDPs and requesting revisions/corrections as necessary, analyzing program hours, 
settling disputes, reminding teachers about deadlines, and answering program-related 
questions. 

In ten of the twelve currently participating districts where we conducted interviews, the 
Career Ladder Review Committee made an annual presentation to the local school board on 
the state of the program. The weight attached to the presentation varied from district to 
district. 

In six districts among the current participant districts in our sample, Career Ladder 
Review Committees were involved in recruiting teachers for the program. Sometimes, 
school building representatives, who may or may not be committee members, participated in 
recmitment activities. Building representatives are teachers selected from each school within 
the district to help answer questions from potential and current program participants. 

In a few districts we studied, the Career Ladder Review Committee was heavily involved 
in authoring the DCLP. In three of the currently participating districts where we conducted 
interviews, the committee held discussions about the program’s current standards and made 
decisions about whether priorities should be revised or requirements increased. In some 
cases, such discussions related to committee members’ authorship of the DCLP. 



II: The Ro/e of the District 




Chapter III 



Teachers’ Experiences 



T his chapter delves into the school districts to explain how teachers come to 
participate in the program, how they become eligible for Career Ladder payments, 
and what they must do to receive the payments. 

A. Teacher Participation 

A district’s decision to participate in the Career Ladder program opens opportunities for 
teachers to join the district’s program voluntarily and become eligible for supplemental pay. 
Teachers must have at least five years of public school teaching experience in Missouri 
before becoming eligible for Career Ladder. Then, they must submit and obtain approval of 
a teacher Career Ladder plan. Next, they must pass an evaluation, typically by a principal, 
called the Performance-Based Teacher Evaluation (PBTE) and, finally, complete and 
document the activities specified in the plan. 

1. Enrollment Process and Timeline 

To be eligible for supplementary pay under the Career Ladder program, teachers in 
participating districts must be serving on a regular-length full-time contract and must have 
Missouri teacher certification; they also must formally enroll in the Career Ladder program. 
To enroll in Career Ladder and qualify for awards, teachers must develop a CDP that 
associates each Career Ladder responsibility with either the teacher’s Professional 
Development Plan or a designated improvement plan such as the DCLP, Curriculum 
Development Plan, School Improvement Plan, Missouri School Improvement Plan, or some 
other instructional improvement. The Career Ladder Review Committee must then approve 
the teacher’s CDP. 
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Teachers must submit CDPs either at the end of the school year preceding participation 
or at the beginning of the school year to which the plan applies. 3 The Career Ladder Review 
Committee may ask teachers for revisions. When prompted, few of our focus group 
participants or district interviewees mentioned any involvement of district staff in CDP 
review. Most teachers felt that the enrollment process was routine, although a few 
mentioned difficulty with the initial process of crafting a CDP and enrolling in the program. 

Forecasting the takeup and completion rate can be a challenge for some districts trying 
to budget for future teacher pay supplements. Partly as a way to forecast Career Ladder 
payments for budgeting, one district in our sample required teachers to sign an “intent to 
participate” form in January as a condition of participation in the program. Unfortunately, 
the rule proved less useful than intended because many teachers completed the form just to 
maintain eligibility, regardless of their true intentions. The district in question has recently 
reduced hours and compensation to half the state-recommended level and has instituted 
more stringent paperwork requirements. 

2. Trends in Teacher Participation 

Participation by Missouri teachers in Career Ladder has been growing, though not as 
sharply as the rate of increase in district participation. Figure III. 1 shows the rates at which 
district participation and teacher participation have increased. (The previous chapter 
discussed district participation trends). Teacher participation nearly doubled, rising from 10 
percent of Missouri’s teachers in 1990 to 18 percent in 2007. The increase in both the 
number and size of participating districts is driving the overall growth of Career Ladder 
teachers. The figure shows a particularly sharp increase in the late 1990s in the number of 
teachers potentially eligible, that is, in participating districts. Not shown, but implied in the 
data, is another trend: Districts that more recently began participating in Career Ladder tend 
to be larger than districts already in the program. However, the take-up rate, that is, the 
percentage of teachers in Career Ladder districts who apply for and receive supplemental pay 
in the program, has been declining, from 52 percent in 1990 to 42 percent in 2007 as these 
larger districts have joined. This decline has been offset by a rise in the number and size of 
participating districts, producing overall only a slight increase in program participation. 

Our analysis of administrative data on teacher receipt of Career Ladder payments from 
1991 to 2007, the years for which data were available, shows that districts adhered to 
program rules. Fewer than 0.1 percent of teachers had been assigned to a Career Ladder 
stage without the statutory minimum years of experience. Table III. 1 shows the percentage 
of participants at each stage by the experience level at which they began receiving payments 
at that stage. While 25 percent of participants who were ever at Stage I began that status 
with 5 or 6 years of experience, the rest had more experience than that when they started. 
The experience level among starting participants reflects the length of time needed to 
qualify, the possibility that the district had not been participating in Career Ladder until 



3 For example, plan submission could extend from April to October 2007 for teachers interested in 
participating in the program during the 2007—2008 school year. 
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participant-teachers had already accumulated considerable experience, and the possibility 
that the program has attracted more experienced teachers to the district. For stage II, 79 
percent had 10 or more years of experience when they started at that stage; for stage III, 
nearly all teachers had the required 1 0 years of experience. 

Figure 111.1. Teacher Participation Rates 

Percentage 




— — Percentage of state's teachers in Career Ladder 

— ■ — Percentage of state's teachers in Career Ladder districts 
Percentage of state's districts in Career Ladder 

- - -X- - - Career Ladder participation rate in Career Ladder districts 



We also examined the duration of each teacher’s stay at a given Career Ladder stage to 
gauge the speed of progress through the program. Of participants who completed their 
careers within the 16-year period between 1991 and 2007, the average stay at stage I was 1.7 
years; for stage II it was 2.1 years; for stage III, it was 5.6 years. 4 



4 



We use those with complete careers in order to avoid undercounting stays cut off by the endpoints of 



the data. 
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Table 111.1. Experience Level at Initiation of Each Career Ladder Stage 


Career Ladder Stage 


Teaching Experience 


Stage 1 


Stage II 


Stage III 


Experience category (percentages) 








Fewer than 4 years 


0.5 


0.1 


0.2 


5-6 years 


24.6 


0.1 


0.1 


7-9 years 


32.1 


20.8 


1.0 


10+ years 


42.9 


79.0 


98.7 


Total 


100.0 


100.0 


100.0 


Average experience level (years) 


10.9 


15.2 


18.6 



Note: Data pertain to Career Ladder participants who began teaching in Missouri after 1990. 



3. Decision to Participate: Do Districts Recruit Teachers? 

Districts are not required to advertise or otherwise reach out to teachers to inform them 
about the program or to encourage eligible teachers to participate. Despite the state 
matching component for program funding, additional teacher participation represents an 
additional district financial burden; thus, for some districts, cost containment may be a 
reason to limit program outreach activities. 

Districts varied in the extent to which they were active in making teachers, particularly 
new hires, aware of the program. Some interviewees reported that they use the program as a 
recmitment tool and advertise the program to potential hires, although this claim was rarely 
reflected in the comments of focus group participants despite our asking explicitly how the 
teachers had been introduced to Career Ladder and what messages they received in 
connection with the program. A number of focus group respondents recalled asking their 
prospective employer about Career Ladder opportunities when interviewing for their current 
position; some were Career Ladder participants in a previous district, and one was advised by 
her professors to ask about the program as she worked toward her education degree. A few 
focus group respondents questioned whether administrators, given their limited involvement 
with the program, could provide prospective teachers with accurate information about 
Career Ladder. 

A consistent message from the telephone interviews with district representatives and 
from the in-person focus groups with teachers is that the program is largely teacher-driven; 
program recruitment and information provision were no exceptions. Word of mouth was 
perceived as a strong force in disseminating program information. Participating teachers, 
particularly Career Ladder committee members or “[school] building representatives” who 
may be responsible for handling questions, provide a large amount of program information. 
They are often involved in contacting eligible or newly hired teachers and informing them 
about the program. A number of districts make a point of mentioning the program during 
teacher staff meetings. Some participating teachers, particularly those involved in program 
oversight, encourage eligible teachers to participate; there was no indication that district 
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personnel encourage participation, aside from possibly advertising the program during the 
hiring process (as already described). 

4. Decision to Participate: Why Do Teachers Participate? 

Career Ladder participants mentioned additional teacher compensation as the primary 
motivating factor for program involvement. Many noted that the Career Ladder payment 
covers work or activities that teachers were already performing or would perform regardless 
of payment under Career Ladder (although, as required by the state, the activities must be 
performed outside of contracted teacher time). Some teachers noted that an additional 
incentive was the opportunity to perform work that they previously wanted to perform but 
were reluctant to undertake because of a lack of compensation. One district places a strong 
emphasis on student tutoring across the district, not just through the Career Ladder 
program. However, the average hourly wages under Career Ladder for tutoring are 
considerably higher than the wages paid by the district to non-participating teachers. 

Except for the two districts in the state with independent pension programs. Career 
Ladder payments counted toward base salary for the purposes of determining defined 
benefit pension payments under the state’s teacher pension plan. This arrangement makes 
Career Ladder payments further desirable, particularly for teachers close to retirement who, 
given experience requirements, are likely to receive the highest level of compensation under 
the program (discussed later). 

5. Possible Barriers to Participation 

Focus group participants consistently cited paperwork as a barrier to participation, 
particularly in some districts more than others. The basic Career Ladder paperwork can be 
lengthy, and the process of documenting hours, which some respondents observe as 
increasingly stringent in recent years, may be time-consuming. In some cases, districts’ 
rigorous documentation requirements present an additional burden. In an attempt to 
document achievement growth, one district required student who were being tutored under 
the program to take pre- and post-tests. These fixed costs of program participation that 
cannot count as Career Ladder activities for the purposes of compensation lower the average 
compensation under the program. 

B. Qualifying for and Receiving an Award 
1. Advancement Along Career Ladder 

Advancement to each of Career Ladder’s three stages depends on a combination of 
years of teaching experience in Missouri, experience at the previous stage, and classroom 
performance. Table III.2 presents the state minimum qualification criteria for each stage of 
the career ladder. For Stages I, II, and III, respectively, the state requires 5, 7, and 10 years 
of teaching in Missouri public schools. The teacher must spend two years at Stage I before 
advancing to Stage II and three years at Stage II before advancing to Stage III. However, 
districts may (and, based on our interviews, do) waive most of these requirements, relying 
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only on experience and a single year at the previous stage. The PBTE, a statewide teacher 
evaluation system, measures classroom performance. Under the PBTE, an administrator 
rates the teacher along 20 criteria. For Career Ladder, a teacher must meet expectations in 
each criterion and, depending on the stage, exceed expectations on some fraction of the 
criteria. 



Table 111.2. Missouri Career Ladder, Summary of Program Features by Stage 





Career Ladder Stage 


Program Feature 


1 


II 


III 


Minimum eligibility requirements 
Experience teaching in Missouri 
Experience at the previous Career Ladder 
stage 3 

PBTE criteria scoring expected or above 3 
PBTE criteria scoring above expected 3 


5 years 
N/A 

1 00% 
0% 


7 years 
1 year 

1 00% 
10% 


1 0 years 
1 year 

1 00% 
15% 


Minimum work requirements 
Number of Career Ladder responsibilities 
Hours of work (minimum) 


2 activities 
60 hours 


3 activities 
90 hours 


4 activities 
120 hours 


Bonus Payments 
Maximum bonus amount 
Implied hourly rate (maximum 
bonus/minimum hours) 


$1,500 
$25. 00/hour 


$3,000 
$33. 33/hour 


$5,000 
$41 .67/hour 



Notes: 

a Minimum years at previous stages for Stages II and III are 2 and 3 years, respectively, but 
waivers are offered for teachers with 7 and 10 years of experience, respectively. Given that 
those experience levels are already minimums for each stage, the waivers would be the norm. 

b PBTE stands for Performance-Based Teacher Evaluation. 



Teachers with National Board Certification (NBC), a voluntary advanced teaching 
credential offered by the National Board for Professional Teaching Standards, are 
automatically eligible to receive compensation under Stage III without any extra work. If a 
district does not participate in Career Ladder, an NBC teacher may petition the district to 
participate on a limited basis so that he or she may receive Career Ladder Stage III 
compensation. 

At their discretion, districts may enact additional requirements at each stage per DESE 
regulations. Districts in our sample did so in a number of cases; some districts required 
teachers to complete graduate coursework at all stages or to work toward or attain a master’s 
degree in order to move to Stage III. One district required one year of instmction in the 
district before participation regardless of any previous experience. Another district enacted 
more stringent requirements for performance on the PBTE. Districts also lowered 
requirements in several cases, specifically in terms of years of experience. Two districts 
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allowed teachers to waive one or more years of participation at Stage I or II if they possessed 
the requisite years of overall teaching experience either in Missouri or in general. 

2. The Performance-Based Teacher Evaluation 

Teacher performance, as measured by the PBTE, is an important component of Career 
Ladder, at least as designed by the state. In practice, the influence of performance 
evaluations on Career Ladder eligibility and advancement depends on the district and school 
principal. 

The PBTE is designed to provide administrators and teachers with information and 
feedback regarding teaching competence and to help teachers improve on a continuing basis. 
Teachers in each district are rated on 20 criteria spanning the following six areas: (1) 
engaging students in class, (2) correctly assessing students, (3) exhibiting content knowledge, 
(4) professionalism in the school, (5) participation in professional development, and (6) 
adherence to the district’s education mission. 

As applied to Career Ladder, each eligible teacher must show evidence of performance 
at or above the expected level on each of the 20 rating criteria in order to maintain program 
eligibility. DESE regulations stipulate that qualification for Stages II and III must also 
involve performance above the expected level on 10 and 15 percent of the criteria, 
respectively, on the most recent PBTE. In addition, state guidelines for the program require 
DCLP to outline procedures for PBTE evaluator training, covering all of the main areas of 
the PBTE as well as consistency and reliability among evaluators. 

Districts may, at their discretion, enact additional requirements at each stage, as 
provided under DESE regulations. Regarding the PBTE, one district in our sample 
required teachers to achieve a higher percentage of evaluation criteria above the expected 
level in order for teachers to qualify for Stages II and III (15 and 35 percent, respectively). 

In the absence of a direct linkage between the evaluation ratings and student 
achievement, the PBTE guidelines and process represent an attempt to formalize what is 
inherently a subjective activity. The frequency of the evaluation process varies by teacher 
experience; non-tenured teachers are subject to annual reviews, whereas tenured teachers are 
reviewed on a five-year cycle. Teachers attain tenure after five years of teaching; accordingly, 
the evaluation process as applied to tenured teachers is the only one applicable to Career 
Ladder participants. According to the PBTE guidelines, all teachers should participate 
annually in professional development activities. 

For tenured teachers, the evaluation process involves a minimum of one scheduled and 
one unscheduled classroom observation, along with professional development documents, 
work sampling (such as lesson plans), drop-in observations (shorter, more informal visits 
than classroom observations), and other materials such as lesson plans, with all data 
collected for the evaluation recorded and filed. An administrators meets with the teacher 
before (in the case of scheduled observations) and after observations. The administrator 
then develops a formal report that incorporates all data collected during the evaluation 
process and outlines the teacher’s performance in relation to PBTE standards and criteria. 
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Districts are directed to develop a formal review and appeal process. The evaluation is 
designed to be ongoing throughout the school year, with periodic progress evaluations and 
criteria assessment. 

The state PBTE materials represent guidelines for districts, not a formal mandate. Most 
districts we contacted, however, used the state forms and guidelines, with some notable 
variation and with one exception. The majority of respondents indicated that tenured 
teachers complete the PBTE every three years as opposed to every five years. Even though 
the DESE guidelines stipulate that PBTE performance should be measured from the most 
recent final evaluation for the PBTE, some indicated that teachers would need to undergo a 
formal evaluation if they planned to qualify for or advance in Career Ladder in the following 
year. 



Both focus group participants and district interviewees frequently mentioned 
collaboration with administrators on PBTE components that influence Career Ladder 
eligibility or advancement. Some teachers told us they chose when the scheduled 
observation would occur and what lesson content would be observed. Some also stated that 
they suggested or identified areas in which they thought they would or should achieve a 
rating of above the expected level. District interviewees mentioned a number of 
accommodations for Career Ladder teachers; one refrain was that administrators wanted to 
avoid confrontations over PBTE ratings and thus would ensure that Career Ladder teachers 
met benchmarks for advancement, sometimes by correcting a poor evaluation or completing 
the PBTE with an eye toward the list of Career Ladder participants. However, some 
respondents cited a more pro-active approach of addressing problems before any conflict 
arose over Career Ladder eligibility. 

According to focus group participants, administrators did not apply the PBTE 
guidelines uniformly. Teachers raised the concern that administrators’ personal opinions 
about teachers could affect ratings. For example, teachers’ accounts of their administrators’ 
definitions of and approaches to the “above expected” level varied considerably between 
districts; some respondents were told by administrators that the rating pertained only to 
exemplary work while other administrators rated all teachers above the expected level. One 
teacher said, “Eve known administrators to hold [the evaluation rating] over people’s heads.” 
Another said, “I think [the administrator] knows if you’re doing your job, and your 
evaluations have been fine they just sign off on it and assume you’re good to go for Career 
Ladder.” 

We also discovered examples of lax or nonexistent adherence to the state guidelines on 
the PBTE. One district in our sample stopped following the state PBTE guidelines several 
years ago according to our interview respondent. Instead, the school administrator simply 
recommends and approves teachers for Career Ladder participation. 

3. Award Levels 

State guidelines set a maximum level of compensation under Career Ladder: $1,500 for 
Stage I, $3,000 for Stage II, and $5,000 for Stage III. Districts may designate payment 
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amounts lower than the state guidelines; however, “each stage shall contain responsibilities 
commensurate and adjustable to the compensation offered for that stage.” Therefore, the 
level of effort for each stage in terms of hours of work must be proportionally adjusted to 
reflect payment amounts lower than the state maximums. Districts may offer more than the 
state maximums, with the district paying 100 percent of the amount over the state 
maximum. 

Among the districts we studied, the timing of payments to teachers varied. State 
payments to districts are made on July 20 as part of the normal disbursement to local 
districts of state education funds earmarked to underwrite Career Ladder activities from the 
preceding school year. 5 Some districts distribute the state and local portions of the teacher 
bonuses separately, with the district-funded portion paid before July 20; other districts make 
one lump payment to participating teachers, sometimes before receiving payment from the 
state on July 20. 

The compensation for each stage under the program has not been revised since the 
original authorizing legislation took effect in the 1986-1987 school year. As a result, the real 
value of the benefits has steadily eroded due to inflation. For example, the maximum payout 
of $5,000 had the same purchasing power in 2007 as $2,683 in 1987, a reduction in value of 
54 percent. 6 

4. Work Hour Requirements 

Each Career Ladder stage involves a prescribed minimum number of work hours that 
correspond to the maximum payment amounts previously described. The amounts are 60, 
90, or 120 hours for Stage I, II, or III, respectively. For the 2006-2007 school year, the 
average number of hours spent for Stage I, II, and III teachers was 79, 109, and 144, 
respectively. 7 The hours correspond, assuming maximum compensation under the law, to 
supplementary pay of approximately $19, $28, and $35 per hour, respectively. 

As of May 1 of each year, teachers may start logging hours for the next school year, 
although districts may amend activity start and end dates within this framework. In some 
cases, the date after which a teacher may start completing hours will fall ahead of the 
deadline for submitting a CDP, particularly in districts that require the CDP at the start of 
the school year. Nominally, under DESE regulations, teachers may not start logging hours 
for the next school year until they have completed all their previous year’s activities, 
including the Career Ladder Review Committee’s documentation review. 8 



5 Rule 5 CSR 80-850.030 § 8. 

6 We used the consumer price index for urban consumers nationally. The U.S. Department of Labor 
does not publish a reliable mral price index based on goods purchased in Missouri over the same 20-year 
period. 

7 Missouri Career Ladder Annual Report 2006—2007, DESE. 

8 Career Ladder Frequently Asked Questions, DESE. 
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5. Activities Under Career Ladder 

To be allowable under Career Ladder, activities performed by teachers must align with 
what the district has outlined as acceptable activities in its DCLP. A DCLP generally 
outlines the percentage of time to be devoted to student contact, allowable credit amounts 
for professional development activities, and hour limits for particular activities; it also 
prescribes methods for documentation. Some districts may emphasize certain types of 
responsibilities. In general, though, most plans appear to be relatively similar, closely 
following DESE’s model career plan. 

As with increasing requirements for performance under the PBTE, each successive 
Career Ladder stage incorporates a greater number of types of responsibilities or activities 
(two, three, or four activities for Stage I, II, or III, respectively). The requirements ensure 
that teachers expand and diversify the types of work they perform under Career Ladder as 
they progress in the program. DESE also recommends that districts set hour limits on 
certain activities that can easily extend into a commitment of time that, after a point, does 
not directly benefit students, such as Internet research or professional reading. All activities 
must be co-curricular in nature and related to existing priorities as outlined in district or 
teacher curriculum, development, or improvement plans. 

Teachers must submit documentation of their Career Ladder activities to the Career 
Ladder Review Committee for verification and subsequent payment approval. The 
committee must compare teacher activity to the requirements of the teacher’s stage and the 
activities outlined in the teacher’s CDP. Almost all Career Ladder teachers receive their 
supplementary pay. Some respondents mentioned that committees have recently become 
more stringent in their requirements for documenting activities under Career Ladder. Many 
also noted that the state and/or district had recently moved to target more narrowly 
allowable activities under Career Ladder in order to focus more sharply on student 
achievement. 

Some districts allow teachers to amend their plans during the school year if 
responsibilities or overall activity levels change; in theory at least, teachers may change stages 
during the year and/or may reapportion the responsibilities they outlined on their CDP to 
reflect activities completed. 

According to focus group members and interview respondents, school districts placed 
emphasis on activities involving student contact; in fact, DESE recommends but does not 
mandate that one-third of time spent on Career Ladder should involve direct contact with 
students. Respondents mentioned tutoring as their primary Career Ladder work and 
frequently noted student clubs and enrichment activities. Other, slightly less frequent 
activities included workshop attendance, serving on various committees, and curriculum 
development. Respondents also mentioned professional development, particularly in 
districts with graduate hour or degree requirements. One district interviewee mentioned that 
its tutoring requirements provided a way to help fund tutoring activities mandated under No 
Child Left Behind. 



Ill: Teachers ’ Experiences 




C H APTE R I V 



Possible Effects of Career Ladder 



O nce basic program operations are understood, it is natural to seek answers about 
what type of impacts Career Ladder might have had on students and teachers. This 
chapter reports on teachers’ perceptions of the likely consequences of Career 
Ladder. It does not present estimates of program impacts based on objective data, although 
we did ask teachers in focus groups to be candid about whether and how they thought the 
program made a difference. We wanted to understand possible mechanisms by which the 
program might lead to positive (or unintended negative) outcomes. 

A. Direct Effects on Students 

Not surprisingly, focus group participants were confident that Career Ladder had 
considerable benefits for students. The main perceived benefit was improved student 
achievement. Their comments suggested that the program can raise student achievement 
through a combination of at least two possible routes. The most direct route is through 
Career Ladder activities themselves, principally tutoring, helping students learn. Regardless 
of whether the content of the tutoring promotes learning during the sessions, teachers 
reported that the time they spent with students outside of regular classroom hours was a 
useful way to become better acquainted with student needs and interests, making participant- 
teachers more effective with those students during the regular school day. Several teachers 
believed that the program disproportionately benefits at-risk students, who may be more 
likely to avail themselves of after-school activities such as tutoring and clubs. 

Teachers in Career Ladder districts spent time in activities that they believed were 
productive; of course, even worthwhile activities do not guarantee a positive impact on 
students. The net impact that is most relevant for policymaking is the difference between the 
outcomes under Career Ladder and those that would have been realized in the absence of 
the program. According to DESE regulations, acceptable responsibilities are those that 
“exceed the norm for the profession;” 9 that is, activities must take place outside of 
participants’ contracted time such that, according to statute, Career Ladder activities must 



9 Career Ladder Frequently Asked Questions, DESE. 
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not be part of a teacher’s normal responsibilities. Does the availability of payments to 
teachers under the program lead to new activity, or would teachers undertake new activities 
even in the absence of the program? 

Career Ladder teachers who participated in each of the focus groups stated that the 
program’s primary purpose was to compensate teachers for work they were already 
performing or would otherwise perform. In other words, participants suggested that the 
activities they performed under Career Ladder were activities that they would have 
undertaken even if they had not been in the program. They perceived Career Ladder as 
simply a salary increase. If that were the case for all teachers and all activities, then the 
program would be expected to have no direct impact at all, although it could produce 
indirect impacts, as discussed below, resulting from the beneficial effects of higher salaries. 
In fact, one district administrator we interviewed for the study described the development of 
a “Career Ladder mentality” among participating teachers as a hidden cost of the program. 
He felt that teachers often would not perform activities that did not count toward their 
Career Ladder hours whereas they might have performed them before introduction of 
Career Ladder. 

Focus group respondents said that they had no difficulty in meeting the hour 
requirements under Stage I but found the hour requirements under Stages II and III 
challenging. Furthermore, given that the number of responsibilities assumed by teachers 
increased with each stage, teachers had to diversify the types of activities they conducted as 
they progressed through the program. Some teachers already meeting the Stage III hour 
requirements but not yet on that stage because they had not qualified on other measures, had 
to think of still other ways to diversify their activities to progress to that stage. 

Some teachers did mention activities that, in their opinion, would not take place without 
compensation under Career Ladder, such as extended enrichment activities that occur after 
the end of the school year. Many also observed that the amount of tutoring and number of 
after-school clubs had increased as a result of Career Ladder. Several focus group 
respondents reported that the work under Career Ladder represented a combination of old 
and new activities. Therefore, it would seem that, even though the program primarily 
compensates teachers for activities that many might otherwise perform, the aggregate 
amount of work completed by participating teachers may have increased; in fact, some 
activities, particularly the more atypical or burdensome programs such as summer 
enrichment, would probably not have taken place if not for Career Ladder. Further research 
using more systematic sampling and measurement is needed in order to make generalizable 
statements about the amount of work completed by Career Ladder participants. 

B. Indirect Effects 

Regardless of whether the Career Ladder activities had direct impacts on student 
achievement, the program may produce beneficial impacts through two indirect routes. One 
is by making teachers better at what they do. We refer to this as a productivity effect. The 
other is by improving the average quality of teachers in the district through recruitment and 
retention. We refer to this as a composition effect. 
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As for the productivity effect, teachers reported that the quality of their teaching 
improved as they acquired new skills and undertook professional development, particularly 
in districts that required graduate school hours for program participation. Few, if any, 
however, mentioned any impact of curriculum development or improvements in course 
content, which are areas that Career Ladder is ostensibly designed to target. 

1. Effects on Teacher Career Decisions 

The other indirect route for Career Ladder to improve teaching and learning is through 
the composition effect, whereby good teachers are more likely to enter and remain in 
participating districts. Retention effects are likely to arise through increased teacher morale 
and job satisfaction. Most participants we spoke with said that the program improves 
teacher satisfaction, partly because of the increased compensation. Some participants said 
that the program activities increased their connection with students. Many thought that they 
learned and grew from the professional development process and, in some cases, from the 
other activities as well. They also mentioned improved staff cohesion as benefits. 

Few of the focus group participants that we asked said that Career Ladder had any 
impact on their decision to remain in the teaching profession. Most felt that Career Ladder’s 
compensation was too low to have any effect. Others stated that they intrinsically enjoyed 
teaching and that additional compensation was not a factor in their overall career path. 

Participants more commonly said that Career Ladder affected their decisions about where 
to teach. Several long-serving teachers explained that the lack of Career Ladder in a 
neighboring district was a factor in their decision not to transfer. Newer teachers (to either 
the profession or the district) also offered stories suggesting a positive recmitment effect. A 
few teachers said that they learned of the program during the interview process, and others 
who were aware of the program made a point of asking about it when they were making 
employment decisions. 

District interviewees cited retention effects, along with increases in student 
achievement, as program benefits. The district staff interviewed for the study outlined 
benefits similar to those cited by the teachers in focus groups. They viewed increased 
compensation and teacher satisfaction as resulting in improved teacher retention. 
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Chapter V 



Conclusions 



T his report set out to profile the Missouri Career Ladder program by delving beneath 
the surface and comparing how the program works in practice (based on stakeholder 
accounts) with how it works in theory (based on published statutes and regulations). 
We found that in many respects the program operates as one might expect. Districts 
implement rules that tend to follow the state guidelines, with some local tailoring. Teachers 
of a given experience level must perform a set of allowed activities in order to receive 
payments at one of the three Career Ladder levels. 

While Career Ladder provided opportunities to supplement salaries by offering extra 
pay for extra work, in most cases it would not qualify as a performance incentive program 
for teachers. The use of an evaluation tool (the PBTE) to promote teachers along the career 
ladder suggests that it should provide incentives for teachers to improve their practice. 
However, we found little evidence that the PBTE is applied rigorously or implemented 
uniformly in a way that would motivate teachers to raise their performance. Instead, Career 
Ladder payments were seen as a reward for longevity and completion (with documentation) 
of a set of allowable activities such as tutoring, after-school activities, or approved 
professional development. 

We also sought to explain how districts and teachers came to be in the program. We 
found that district size matters. We documented the preponderance of small districts, which 
happen to be mral and poor, in the program despite the rise over time in the participation 
rates for medium and large districts. Larger districts in particular had low takeup rates, i.e. a 
low percentage of teachers who had submitted a Career Ladder Plan and received Career 
Ladder payments. A possible explanation for low takeup rates within a participating district 
might relate to the generosity of the program; Career Ladder benefits have lost nearly half 
of their value to inflation. In the large cities of St. Louis and Kansas City, where the benefits 
do not count toward retirement pay, the incentive is even weaker. Thus the levels may not 
be high enough for many teachers to justify completing the individual plans. Another 
obvious explanation would be related to years of experience and eligibility. Some districts 
may be eager to help their first-, second-, and third-year teachers, but the Career Ladder 
program does not apply to these early career teachers, since they would not have qualified. 
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Finally, a goal of the analysis was to identify the mechanisms by which Career Ladder 
might produce positive impacts. We identified plausible routes whereby the program could 
raise student achievement, improve teacher satisfaction/morale, and improve teaching via 
productivity and composition effects. Teachers reported that they get to know their 
students better as a result of tutoring them outside of class. They also reported that a 
district’s participation in Career Ladder might influence them to choose to teach in that 
district over another district that did not have the program. 

The activities that teachers in engage in, such as professional development workshops 
and student clubs, may be productive in terms of improving teaching and learning. 
However, there is reason to believe that the net impacts would be small as well. For 
example, several teachers confided to us that the activities performed under Career Ladder, 
at least those most commonly performed for Stage I, are similar to what the teacher would 
be doing regardless of the program. 

Ultimately, the question of impact is an empirical one, which can and should be 
answered using the most rigorous quantitative methods available. The companion reports 
by Booker and Glazerman (2008a; 2008b) attempt to answer these questions about impact, 
albeit using existing data and relying on statistical adjustment to generate valid inferences 
about impacts. Nevertheless, the current report and its companions go a long way toward 
describing the Missouri Career Ladder and setting the stage for discussions of policy changes 
and for further research. 



V: Conclusions 
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Data and Methods 



F or this report we consulted existing documents and collected primary data. The 
existing documents included state legislation, rules and regulations issued by DESE, 
and district-level documents such as District Career Ladder Plans, many of which are 
available on school district Web sites. Additional data for the study came from focus groups 
with Career Ladder teachers and structured interviews with district program administrators, 
state policymakers, and stakeholders. We also analyzed a longitudinal database on teacher 
and district participation in Career Ladder that we compiled from data provided by DESE. 

A. Program Documents and Administrative Data 

The first step in our study of the Career Ladder program was to review the state 
materials and legislation governing the program’s operations and structure. The Career 
Ladder was first introduced in 1985 as part of the Missouri Excellence in Education Act, 
known as House Bill 463 and implemented in the following year. The program underwent 
some significant changes in 1996 under Missouri Senate Bill 795. We reviewed the text of 
both laws to describe the historical changes in the program, particularly as related to its 
funding structure. We also examined DESE’s program mles delineated in the Missouri 
Code of State Regulations. The regulations provide additional guidance for districts 
interested in participating in the program, particularly in areas not addressed by the 
legislature, and set a timeline for the application process. 

DESE publications are an additional source of useful program information. One such 
resource is a District Career Ladder Plan that DESE is required by state statute to provide as 
a template for local districts. These and other materials are available online. Web resources 
also include guidelines for teachers and districts, Career Ladder “frequently asked questions”, 
sample program forms, a list of activities inappropriate for Career Ladder, and web-based 
Career Ladder reporting forms for teachers and districts. 

DESE also provided us with electronic files that we used to constmct longitudinal 
databases that proved useful in several ways. For example, we used data on district 
participation patterns and other characteristics to draw samples of districts for the focus 




A-2 



groups and telephone interviews. We also analyzed the state databases to measure district 
and teacher participation rates and trends over time. 

B. Interviews 

We selected a random sample of school districts in which to conduct telephone 
interviews that were aimed at providing the district perspective on the operation of the 
Career Ladder program and the district decision to participate (or end participation) in the 
program. 

1. District Selection 

We identified 16 school districts in which to conduct interviews, 15 of which had been 
participating in Career Ladder during the 2006-2007 school year and one drawn from a list 
of school districts that had ceased program participation within the last four years. We 
stratified the sample in several ways. 

First, we divided the districts on the basis of their Career Ladder participation history: 

Recent Entrants: Districts that began participating in Career Ladder during the 1 999 — 

2000 school year or thereafter. One of our goals for the study was to understand 
why districts participate in the program; recent entrants to the program would be 
best positioned to provide us with such information. We selected four districts 
from the cluster of recent entrants. 

Long-Term Participants : Districts that had participated continuously since the 1 995 — 

1 996 school year. The continuation of participation is another question of interest, 
and long-term participants appeared most able to provide relevant information. 
Operations in these districts may have settled into a predictable pattern, providing a 
more accurate picture of resulting program characteristics. Initially, we sampled 10 
districts from this group. 

Second, we stratified by urbanicity by using the National Center for Education 
Statistics locale codes from the Common Core of Data. Such stratification promised 
representation from all urbanicity categories, ensuring that we would capture variations in 
Career Ladder operations in different types of districts in terms of matching rates. The 
standard measure of urbanicity is also based on the size of the locality and thus was highly 
correlated with district size (e.g., enrollment). Therefore, we did not need to stratify 
explicitly on district size. The stratum (group) definitions were: 

1. Large city 

2. Medium-sized city, urban fringe of medium-sized city, urban fringe of large city 

3. Large town, small town, rural 
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The two district types and three urbanicity categories define six possible combinations. 
We randomly selected districts within each of the combinations in order to get a diversity of 
districts and then added two more districts based on their participation patterns: 

Districts with Odd Participation Patterns-. Districts with an unusual or unexpected 
distribution of teachers across stages were of interest because such districts may 
provide insight into program adaptations. In the case we examined, approximately 
98 percent of teachers were in Stage III of the program. 

Recent Deavers-. Districts that ceased participating in the program since the 2002- 
2003 school year were of interest. We randomly selected one such district, which 
had stopped participating in Career Ladder after the 2004-2005 school year. 

For each district, we identified one or two potentially knowledgeable respondents. 
Although we initially expected that interviewees would be district administrators, we were 
directed in most cases to the teachers responsible for overseeing the program through their 
work on the Career Ladder Review Committee. 

We tailored the interview protocol (shown in Appendix B) to each type of district. For 
example, we asked staff from districts that had recently left the program about why they 
ended their participation. We also divided interviewees into two categories: those with 
knowledge of current program operations (e.g., staff who currently oversee the program) and 
those with historical knowledge of the district’s decision to participate. In some cases, one 
individual fit both categories. In other cases, we conducted two or three interviews in the 
same district. 

2. Completion Rates 

We completed interviews with program administrators from 13 of the 16 districts we 
selected (82 percent); we also completed two state -level interviews. One rural district 
declined participation in both the district interviews and the focus group. We interviewed 
more than one respondent in several districts. The result was 23 completed interviews: 6 
districts in which we interviewed one person, 6 districts in which we interviewed two people, 
one in which we interviewed three people, one interview with a state union leader, and one 
interview with a DESE official. 

C. Teacher Focus Groups 

To gain teacher perspectives on the program, we conducted four focus groups with 
Career Ladder participants, each in a separate type of school district defined by district size 
and urbanicity. Teacher focus groups were designed to help us understand teachers’ 
decisions to participate in Career Ladder, the incentives offered by the program, and 
teachers’ general experience with and attitudes toward the program. 
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1. Site selection 

From among the initial sample of interview districts, we selected four school districts 
for a focus group with participating teachers. 10 To ensure a diversity of opinions and 
viewpoints, we stratified districts by size and urbanicity, following a method similar to that 
used for the district interviews. We divided districts into categories of large urban, suburban, 
and rural. We restricted the potential rural districts to those with district enrollment greater 
than 300 students, thereby ensuring enough potential participants to make focus group 
recmitment feasible. 11 

With only one large urban district in our sample, we selected that district with certainty. 
To address logistical and cost concerns, we restricted the choice of a suburban district to be 
among the three five districts sampled in close proximity to the large urban district already 
selected and then randomly selected a suburban district from among this subsample. We 
initially selected two mral districts at random from among the sample of interview districts. 
As noted, one of the districts declined to participate in either the district interview or the 
focus group. Subsequendy, we selected a replacement mral district at random from the 
district interview sample. 

2. Recmitment and Completion 

To recmit teachers for the focus groups, we used lists of participating teachers obtained 
from the selected districts. We stratified and randomly selected participating teachers along 
two dimensions: Career Ladder stage (I, II, or III) and grade level of instruction (elementary, 
middle, or high school). Hence, we had nine selection bins and drew an initial sample of two 
teachers from each bin. In one case, we released an additional recruitment sample as 
necessary. 

We vigorously recruited focus group members, producing a sample that was diverse 
with respect to all grade levels, years of experience, and Career Ladder stage. Respondents 
received a $40 incentive payment. Each focus group lasted about 90 minutes, followed an 
identical protocol, and included an average of almost eight respondents. The protocol is 
shown in Appendix C. The average experience level across all focus group participants was 
15 years, ranging from 7 to 29 years. Of those whose Career Ladder stage we could identify, 
23 percent were at Stage I, 46 percent at Stage II, and 31 percent at Stage 3. 



10 Because we sought input from current participants in all stages of the Career Ladder, we excluded the 
district that recently left Career Ladder and the district with nearly all of its teachers in Stage III of the program. 

11 This restriction removed three rural districts from our pool of potential focus group districts. 
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Protocol for D i s t r i c t - L e ve l 
Interviews 



Screening Questions: 

• In what capacity are you involved with the Career Ladder? 

• How long have you been so involved? 

• What other staff members at the district (and/or teachers) are involved with the 
program? 

• [At this point, can direct person appropriately to the different portions, and/or 
ask for additional contact information]. 

To streamline interviews, obtain and review the district’s District Career Ladder Plan before 
calling the district. 



Short-Form Interviews 

If district is long-term participant, complete section A. 

If district is a recent Career Ladder program entrant, complete section B. 
If district is a recent leaver, complete section C. 



A. Long-Term Participants 

The two interview sections may require two respondents, unless the person who 
currendy oversees the program was involved with the initial participation decision. 
If this is not the case, then the interviewer needs to be directed to someone who 
was involved with the initial participation decision to administer section (b). As you 
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will note, there is overlap between the sections. If one person is completing both 

sections, adjust as necessary. 

a. Current Implementation Status 

1. What factors were involved in your district’s decision to participate in the 
Career Ladder program? 

2. Has the district experienced any benefits from participation in terms of student 
achievement or teacher retention? Have there been any other benefits? [Probe: 
If respondent unsure, then say: Such as teacher performance or satisfaction?] 
Have there been any drawbacks? Is this based on anecdotal, qualitative, or 
quantitative evidence? 

3. Why does the district continue to participate? Does the district reevaluate its 
participation in the program? If yes, how often? [Probe: Are there mandated 
frequencies? Are there triggers (match rate change, budgetary issues, student 
achievement) in this regard?] If no, any particular reason why? 

4. The state guidelines provide districts some leeway in a number of areas [Probe: 
Such as how teachers advance, compensation levels, etc.] How has the district 
adapted the program to better fit its needs? 

5. What types of staff or individuals are involved in the operation of the career 
ladder? 

6. Has participating been a largely positive or negative experiences? What do you 
see as the good or bad features of the program? 

b. Historical Factors/Participation Decision 

1. What factors were involved in your district’s decision to participate in the 
Career Ladder program? 

2. What benefits did the district expect to realize when it decided to participate? 
Did the district expect any benefits in terms of: teacher performance, student 
achievement, teacher retention, or teacher satisfaction? 

3. Starting in the 1996-1997 school year, the state matching formula for the 
Career Ladder program changed. What impact, if any, did the change in match 
rates have on your district’s level of funding contribution? Did this change 
force your district to reevaluate its participation decision? Why or why not? 

4. Has participating been a largely positive or negative experiences? What do you 
see as the good or bad features of the program? 
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All of the “Recent Entrants” entered the program in 2000 or later. As such, the 
person who currently oversees the program may or may not have been involved 
with the initial participation decision. The interviewer must determine whether this 
is the case. If so, then both parts (a) and (b) can be administered to this person. If 
not, then the interviewer needs to be directed to someone who was involved with 
the initial participation decision to administer section (b). There is overlap between 
the sections. If one person is completing both sections, adjust as necessary. 

a. Current Implementation Status 

1. What factors were involved in your district’s decision to participate in the 
Career Ladder program? 

2. Has the district experienced any benefits from participation in terms of student 
achievement or teacher retention? Have there been any other benefits? [Probe: 
If respondent unsure, then say: Such as teacher performance or satisfaction?] 
Have there been any drawbacks? Is this based on anecdotal, qualitative, or 
quantitative evidence? 

3. Why does the district continue to participate? Does the district reevaluate its 
participation in the program? If yes, how often? [Probe: Are there mandated 
frequencies? Are there triggers (match rate change, budgetary issues, student 
achievement) in this regard?] If no, any particular reason why? 

4. Has participating been a largely positive or negative experiences? What do you 
see as the good or bad features of the program? 

b. Historical Factors / Participation Decision 

1. Why did the district decide to begin participating in the Career Ladder in Year 
XXXX? [Probe: Teacher interest, availability of funds, expected benefits?] 

2. What benefits did the district expect to realize when it decided to participate? 
Did the district expect any benefits in terms of: teacher performance, student 
achievement, teacher retention, or teacher satisfaction? 

3. Can you describe what events occurred along the way, between the decision to 
participate and implementing the program? Were there any particular 
individuals or groups that were clear drivers in the decision to participate, the 
planning process, and implementation? 

4. Legislation passed, starting with the 1996-1997 school year, which affected the 
program match rates for districts. Did this change in match rates have an 
impact on the district’s decision to participate? 
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5. Was the process of creating the District Career Ladder Plan easy or difficult? 
What level of guidance did you receive from the state? Is there any leeway 
provided in the process? 

6. What have been the reactions of staff to the institution of the program? [Probe: 
if unsure of levels, specify at the district, school, and classroom levels]. 

7. Has participating been a largely positive or negative experiences? What do you 
see as the good or bad features of the program? 

C. Recent Leavers 

For this interview section, it will likely be more difficult to identify the correct 

initial respondent. For the recent leaver district that has not reentered the program, 

we may have to simply ask who previously oversaw the Career Ladder program 

1. Why did the district decide to cease participating in the Career Ladder in Year 
XXXX? [Probe: If unsure, ask: What were the primary factors behind the 
decision? Costs, dissatisfaction, etc.?] 

2. Were there any particular individuals or groups that were clear drivers in the 
decision? Did complaints move up the school hierarchy or was it largely a 
district decision? 

3. Can you provide a timeline of the decision process leading up to exiting the 
program? 

4. Did the district realize any benefits from the program while it was participating? 

5. 2007 re-entrant: At the time you left the program, did you expect to return to the 
program in a certain number of years? What factors were involved in these 
decisions? 

D. Districts with Odd Participation patterns 

1. How does the Career Ladder operate in your district? How does the district use 
the flexibility provided by the state? How is your program unique? Can you give 
an outline of the processes and decision makers involved? 

2. It appears that ALMOST ALL participating teachers in your district are at Stage 
3 . Why is this the case? [Note: If all teachers at Stage 3, ask whether this is due 
to the Stage 1 and 2 exemption for teachers with National Board 
Certifications]. 

3. [If not due to National Board Certification issue]: Do you explicitly limit the 
program to certain teachers, or only publicize for certain groups of teachers? 
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Note: Some of these questions may arise in some of the short form interviews, and 

in those cases we will omit from this script. Those instances are noted as needed. 

A. Teacher Questions 

1. What are the rules for teacher eligibility; is there anything in place aside from 
the state guidelines? 

2. Does the district/ schools publicize the program? E.g. do they actively 
encourage teachers to participate or provide information about the program? 
Has the state undertaken any such activities? 

3. Is the program competitive, either to become a Career Ladder teacher or to 
move up Stages? 

4. Are there barriers to participation for teachers, besides those explicitly outlined 
by the state? [Probe: such as: long application forms, review of their submitted 
documents, etc.] 

5. Is there any aging out of the program? For example, after a certain amount of 
time at Stage 3, do teachers have to leave the program? 

6. How often do teachers fail to meet the criteria outlined for their Career Ladder 
stage, or in their individual Career Development Plan? What possible reasons 
are there for this (certification, failing to meet hour requirement, etc.)? 

7. What is your perception of why teachers enter or leave the program? 

B. Performance-Based Teacher Evaluation 

1. Can you describe how the PBTE process operates in the district? Is there any 
possible variation between schools in how it operates? 

2. What, for your district, is the relationship between the PBTE and the MCL? 
[Note: for long-term participants, we have the District Career Ladder Plan]. 
[Possible probe: Has the intersection of the PBTE and the achievement of 
MCL stage requirements created tension between administrative staff and 
participating teachers?] 



C. Finances 

1. What is your impression of the bonus levels as they are currently set? [Probe: 
Levels are $1500, $3000, and $5000]. When Career Ladder funds were part of 
the school foundation formula (i.e. basic school aid), and the legislature did not 
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fully fund the formula, did you reduce Career Ladder payments accordingly or 
obtain funds to make up the difference? 

2. When are monies distributed to teachers? Upon proof of successful 
completion? Mid-year? Beginning of the school year? 

3. Have taxes been levied (as allowed for under the law) to raise money for the 
Career Ladder Program? 

4. Does the MCL payment count towards teacher salary for the purposes of 

determining defined benefit pension payments? [Note: KC operates 

independent pension plan] 

5. The 1996 legislative change stipulated that the district would have to provide 
the 1995-1996 matching level for teachers who participated during 1995-1996 
and continued to participate in the program. Has this had any significant effect 
for the district? 

6. A 1996 legislative change equalized payments for teachers within each stage 
across districts. This meant that all teachers in the district on the same Career 
Ladder stage received the same amount of award. Did this have any effect on 
teacher participation in the program? Did this change mean that there was any 
true practical change in the payments to teachers? [Only ask: if at district and 
involved in MCL since 1996 legislation change] 



D. Career Ladder Committee and Plan 

1 . Can you give a description of the activities that the Career Ladder committee is 
involved in on a regular basis? [Note: Ask interviewee if they have any 
documents that they can share] . 

2. What is relationship between the local Career Ladder committee and the school 
board? 

3. How are teachers chosen to sit on the committee? 

4. What amount of leeway is given by the state in crafting the District Career 
Ladder Plan? What is your impression of the creation process, i.e. difficult, 
easy? [Note: Asked in the recent entrants interview] 

5. Did the state provide you with a model career ladder plan as an example? 

6. Have yourself and/or other staff attended any workshops or other events 
concerning the Career Ladder program? Does the state or any other entity 
provide technical assistance or other types of guidance? 
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Focus Group Protocol 



A. INTRODUCTION 

My name is Tim Silman and I work for Mathematica Policy Research. Mathematica, an 
independent policy research firm with extensive experience in conducting education 
research. We have received a grant from the Ewing Marion Kauffman Foundation of Kansas 
City to conduct a study of the Missouri Career Ladder program. We are interested in learning 
more about the program and about your experiences as participants. This discussion is your 
chance to let us know how the program really works and how it relates to your career as a 
teacher. 

PRIVACY: Everything you say here is private. We will not release any information that 
could be used to identify you. No individual staff member will be quoted by name. Our 
report will describe the range of views expressed by teachers across districts, but specific 
comments will not be attributed to specific individuals or districts. We hope that this will let 
you be as candid as possible in expressing your opinions. 

CONFIRM DURATION OF THE FOCUS GROUP: 75-90 minutes 

MECHANICS OF TAPING: I am taping our discussion so that I can focus on the 

conversation now and write notes later. No one outside of our research team will have 
access to the tape. It will be helpful if you speak up, speak clearly, and speak one at a time. 

ROLE OF THE MODERATOR: I’m going to lead the discussion. We have a number of 
topics to discuss during the time we have for this discussion. At times, I may need to move 
the conversation along to be sure we cover everything. 

There are no right or wrong answers. People may disagree, and that’s okay. Please feel free 
to offer your opinions, whether positive or negative. 

PARTICIPANT INTRODUCTIONS: To get started, please introduce yourself by telling 



me: 




1. Your first name 
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2. What grade and subject you teach 

3. How long you have been teaching; in total, in MO, and in the district 

4. Your Career Ladder stage or level 

B. PROGRAM PARTICIPATION 

1. How did you first hear about the Career Ladder program? 

Did anyone in your school or the district encourage you to participate in the 
Career Ladder? How did they encourage you? 

How is the program marketed or advertised to teachers? To current teachers or 
prospective teachers? By subject or grade level, or on an individual basis? 

2. Why did you decide to participate in the Career Ladder? What factors did you 
consider when making your decision? 

PROBES: 

The level of compensation? 

The amount and type of additional responsibilities? 

Something else? 

3. [For teachers who started participating at some point after the program was 
implemented]: Was the Career Ladder program something you were aware of 
prior to meeting the eligibility requirements? 

IF YES: 

Did it influence your decision to stay within the district? 

Or to move to a participating district? 

4. Is the number of slots in the program in any way limited? Or can any teacher 
that meets the requirements participate? [COMPETITIVE?] 

5. Why do you feel that eligible teachers do not or would not participate in the 
program? 

6. The [Name] district has been participating in the program since XXXX. Were 
teachers, yourselves included, involved in any way with the district’s decision to 
participate? 
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How, or in what ways? 

C. CAREER LADDER COMMITTEE AND PLANS 

1. Can you provide an outline or timeline of the Career Ladder application 
process? 

Does the first year process differ significantly from that in later years? 

What do you think about the application process? Easy? Difficult? Quick? 
Lengthy? 

2. Tell me about your interactions with the Career Ladder committee. 

PROBES: 

How often are you in contact with the committee? 

Is the committee only involved with reviewing individual career ladder plans? 
Or do they perform other functions? 

3. How are teacher representatives chosen for the Career Ladder committee? 

4. How is your Career Ladder plan evaluated by the committee? Are there any 
factors involved besides the responsibilities/activities it outlines and their 
overall and relative amounts? 

5. What has been your experience with the process of advancing up the Career 
Ladder? 

Have any of you received any sort of waiver for the Career Ladder 
requirements? 

IF YES: 

For which component? 

6. Can you describe the Performance-Based Teacher Evaluation? 

PROBES: 

How does the process work? 

Who is involved in the evaluations? 

How involved are teachers in the process of their evaluations? 
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Are there other evaluations of teachers that you have to undergo? 

7. What are your thoughts on the linkage between the Career Ladder and the 
PBTE? 

PROBE: 

For example, to qualify for the Career Ladder there is a mandated level of 
performance on the PBTE. What is your opinion of this? 

D. ACTIVITIES UNDER THE CAREER LADDER 

1 . How do your Career Ladder activities relate to your Career Development Plan? 

2. How much time during the year do you spend doing Career Ladder activities? 
[How far above the minimum requirements do you end up going in terms of 
hours?] 

3. What types of responsibilities or activities do you complete under Career 
Ladder? 

4. What proportion of your time do you spend in these areas? 

5. How does this vary by Career Ladder stage? 

6. Teachers do a lot of things outside of the classroom. Are the activities you do 
for Career Ladder the types of things you did before participating in the 
program? Or do you do different types of activities now that you participate? 

7. How much are your school administrators involved in the program? 

E. PROGRAM COMPENSATION 

1. Has there been any significant change in the amount of Career Ladder 
compensation over time? 

2. Has the district ever fallen short in the amount of Career Ladder compensation 
provided, compared to what was promised? 

IF YES: 

What was the stated reason? 

3. What is your opinion on the levels of compensation under the Career Ladder, 
given what is required of teachers? 

IF NOT ENOUGH: 
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How far short? A little, a lot, somewhere in between? What do you think would 
be fair? 

F. OPINIONS ON EFFECTS, BENEFITS, AND THE PROGRAM OVERALL 

1. What benefits has the program had for you, besides additional pay? 

Has the program improved your satisfaction? 

Encouraged you to stay in the district, or stay in teaching? 

Anything else? [Teachers in general?] 

2. What benefits has the program had for your students? 

Improved student achievement? 

Student behavior? 

Anything else? [Teachers in general?] 

3. What other benefits have you seen of the program? 

4. What barriers do you see to participating in the program? 

5. I want to return to a concept we touched on earlier. How do you think the 
Career Ladder has influenced the decisions you’ve made with regard to your 
career? 

Has it affected? 

Your choice of district? 

Your effort in relation to the PBTE? 

Your decision on whether to retire, or to stay in the profession? 

6. Has the Career Ladder increased any of the following: 

Time you spend with students? 

Time you devote to professional development? 

Time you spend on certifications or degrees? 

7. What is your overall opinion of the Career Ladder program? 

What elements do you like? 
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What do you dislike? 

How can the program be improved? 

8. Is there anything else you would like to share? 

CLOSING REMARK 

Thank you for coming here to speak with me. You’ve all given me a great deal of 
valuable information that I think will help our study tremendously. Should you have any 
questions, additional thoughts, or concerns please feel free to contact me [pass out business 
card] . 
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